Data Statements for Natural Language Processing
This webpage contains information about data statements for language datasets used in natural language processing systems. The schema elements have been honed to the particular characteristics of language datasets, including speech context, speaker demographic, and annotator demographic. The most recent schema elements (Version 2) are listed here. Detailed definitions of the elements are provided in A Guide for Writing Data Statements, linked below, along with rationale and suggestions for writing each element as well as general best practices. A table summarizing the changes from Version 1 to Version 2 can be found under Other Resources below.
Schema Elements Version 2
- HEADER
- EXECUTIVE SUMMARY
- CURATION RATIONALE
- DOCUMENTATION FOR SOURCE DATASETS
- LANGUAGE VARIETIES
- SPEAKER DEMOGRAPHIC
- ANNOTATOR DEMOGRAPHIC
- SPEECH SITUATION AND TEXT CHARACTERISTICS
- PREPROCESSING AND DATA FORMATTING
- CAPTURE QUALITY
- LIMITATIONS
- METADATA
- DISCLOSURE AND ETHICAL REVIEW
- OTHER
- GLOSSARY
Writing Data Statements
Other Resources
Bender, Emily M. and Friedman, Batya. 2018. Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science. Transactions of the Association for Computational Linguistics 6:587-604.
Data statement samples:
- Data Statement for MuST-SHE
- Data Statement for the Public DGS Corpus
- Data Statement of the Corpus of Basque Simplified Texts
Table for converting from Schema Version 1 to Schema Version 2
LREC 2020 Workshop ‘Data Statements for NLP: Towards Best Practices’
A Short History of Data Statements
Data statements were first conceptualized in 2017 by Emily M. Bender and Batya Friedman at the University of Washington where they were initially developed for language datasets used in natural language processing systems. The first version of data statements was published in 2018 in Transactions of the Association for Computational Linguistics and presented at the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL). The next two years saw significant interest and uptake. With the goals of supporting broader uptake and learning how to make data statements a suitable practice across different research and institutional contexts, in 2020 Emily M. Bender, Batya Friedman, and Angelina McMillan-Major organized a workshop at the 12th Language Resources and Evaluation Conference. The results of this workshop led to an updated schema (Version 2), a set of best practices, and A Guide for Writing Data Statements all released in 2021.
Data statements are a part of an emerging landscape for toolkits about documentation for transparency in artificial intelligence systems, including Datasheets for Datasets, Model Cards for Model Reporting, Dataset Nutrition Labels, Nutrition Labels for Data and Models, FactSheets, and Data Cards.